Interactive Auditory Display to Support Situational Awareness in Video Surveillance

A key element for efficient video surveillance is situational awareness. Characteristics of human perception (e.g., inattentional blindness) as well as surveillance practice (e.g., CCTV operators have multiple responsibilities) often hinder comprehensive visual recognition of the activities in the monitored area. We support situational awareness and reduce the workload of CCTV operators by complementing the video display by an auditory display. Trajectories of moving objects extracted from surveillance video are sonified by auditory icons. These icons are interactively assigned by the user to each object category of the video and, in this way, form a sonic ecology. We use a spatial auditory display to represent location, direction and velocity of a trajectory with respect to a virtual listener. This facilitates orientation in virtual auditory space in a natural and realistic way that meets users’ expectations. Modification areas are introduced to allow the users to define areas in which auditory icons are modified to further improve situational awareness. We put emphasis on efficient interaction between users and the auditory display to adjust the system according to the monitored area. Finally, we evaluate our approach by a user study and discuss benefits and shortcomings of the proposed sonification in the light of psychology, cognitive science, and neuroscience.